Increased Confidence in Deduplication of Drug Safety Reports with Natural Language Processing of Narratives at the US Food and Drug Administration

The US Food and Drug Administration (FDA) receives millions of postmarket adverse event reports for drug therapeutic biologic products every year. One the most salient issues with these submissions is report duplication, where an experienced by one patient reported multiple times to FDA. Duplication has important negative implications data analysis. We improved optimized existing deduplication algorithm that used both structured free-text data, developed a web-based application support processing, conducted 6-month dedicated evaluation assess potential operationalization process in Comparing predictions reviewer determinations duplicates twenty-seven files case series reviews (with median size 281 reports), average pairwise recall precision were equal 0.71 (SD ± 0.32) 0.67 0.34). Overall, reviewers felt confident about expressed their interest using it. These findings review as supplement human review.